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Abstract 

Background: Common measures of surgical quality are 30-day morbidity and mortality, which poorly describe 
breast cancer surgical quality with extremely low morbidity and mortality rates. Several national quality programs 
have collected additional surgical quality measures; however, program participation is voluntary and results may 
not be generalizable to all surgeons. We developed the Breast Cancer Surgical Outcomes (BRCASO) database to 
capture meaningful breast cancer surgical quality measures among a non-voluntary sample, and study variation in 
these measures across providers, facilities, and health plans. This paper describes our study protocol, data collection 
methods, and summarizes the strengths and limitations of these data. 

Methods: We included 4524 women >18 years diagnosed with breast cancer between 2003-2008. All women with 
initial breast cancer surgery performed by a surgeon employed at the University of Vermont or three Cancer Research 
Network (CRN) health plans were eligible for inclusion. From the CRN institutions, we collected electronic administrative 
data including tumor registry information, Current Procedure Terminology codes for breast cancer surgeries, surgeons, 
surgical facilities, and patient demographics. We supplemented electronic data with medical record abstraction to 
collect additional pathology and surgery detail. All data were manually abstracted at the University of Vermont. 

Results: The CRN institutions pre-filled 30% (22 out of 72) of elements using electronic data. The remaining 
elements, including detailed pathology margin status and breast and lymph node surgeries, required chart 
abstraction. The mean age was 61 years (range 20-98 years); 70% of women were diagnosed with invasive ductal 
carcinoma, 20% with ductal carcinoma in situ, and 10% with invasive lobular carcinoma. 

Conclusions: The BRCASO database is one of the largest, multi-site research resources of meaningful breast cancer 
surgical quality data in the United States. Assembling data from electronic administrative databases and manual 
chart review balanced efficiency with high-quality, unbiased data collection. Using the BRCASO database, we will 
evaluate surgical quality measures including mastectomy rates, positive margin rates, and partial mastectomy re- 
excision rates among a diverse, non-voluntary population of patients, providers, and facilities. 



Introduction 

The Institute of Medicine report "Crossing the Quality 
Chasm", emphasized high quality care should be safe, 
timely, effective, efficient, equitable, and patient-centered 
[1]. The common measures of surgical quality are 30- 
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day morbidity and mortality, which are not ideal metrics 
for breast cancer procedures that have extremely low 
morbidity and < 0.5% mortality. These measures only 
reflect outcomes of surgery, but do not consider the sur- 
gical process or institutional structure - other aspects 
that can influence the overall procedure quality [2]. Sev- 
eral experts have agreed that the quality of cancer sur- 
gery is currently the "black box" of cancer care, subject 
to wide variability at provider-, facility-, and regional- 
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levels, emphasizing the need for new standard, quality 
measures [3]. 

The University of Vermont Breast Cancer Surgical Out- 
comes (VBCSO) database was developed in 2003 to estab- 
lish the feasibility of measuring potential breast cancer 
surgical quality indicators [4], The VBCSO database 
included 910 women diagnosed with breast cancer 
between April 1, 2003- March 30, 2008 who underwent 
initial breast cancer surgery at a single institution. Trained 
nurses abstracted medical record data on initial mastect- 
omy rate, close and positive margin rates, number of sur- 
geries, number of lymph nodes obtained during sentinel 
node biopsy and/or axillary dissection, use of intraopera- 
tive pathology assessment for sentinel nodes, and days 
between diagnosis and initial surgery. Subsequently, these 
data have demonstrated variation in surgical outcomes 
among surgeons who practice at one institution [4]. 

We extended the VBCSO database to three Cancer 
Research Network (CRN) health plans to develop a 
large, non-voluntary, multi-institution database to study 
breast cancer surgical quality. The CRN is a consortium 
of 14 non-profit research centers based in integrated 
healthcare delivery organizations within the HMO 
Research Network [5]. The purpose of this paper is to 
describe the combined Breast Cancer Surgical Outcomes 
(BRCASO) research database, structure, data elements, 
and study methods in detail, including important lessons 
learned in collecting surgical outcomes data. We plan to 
use these data to evaluate variation in surgical quality 
measures (initial mastectomy rate, initial partial mastect- 
omy positive margin rate, and initial partial mastectomy 
re-excision rate) by patient-, surgeon-, facility-, and 
region-level factors. 

Methods 

Data from the VBCSO database were combined with data 
from three CRN health plans (Group Health Cooperative, 
Kaiser Permanente Colorado, and Marshfield Clinic) to 
form a single, large, multi-site database called BRCASO, 
which resides at the Van Andel Research Institute 
(VARI). All study procedures were approved by Institu- 
tional Review Boards at the University of Vermont, 
VARI, and Kaiser Permanente Colorado (through coop- 
erative review for the CRN health plans); we received a 
waiver of consent to collect patient and provider-level 
data. The waiver was granted based on the fact that this 
study was a retrospective review of standard treatment 
data and of minimal risk to the patients; all identifiable 
information was kept at local sites, and only a deidenti- 
fied data set was shared for analysis and reporting. 

Study population 

A total of 6095 women >18 years old were diagnosed 
with stage 0-IV breast cancer between January 1, 2003 



and December 31, 2008. To be eligible for inclusion in 
the database, women had to have their initial breast can- 
cer surgery performed by a surgeon employed by the 
University of Vermont or one of the CRN health plans 
(including CRN health plan-contracted facilities). We 
included all women with ductal carcinoma in situ 
(DCIS), invasive ductal (IDC) or lobular carcinoma 
(ILC); all other breast pathologies were excluded. We 
also excluded males, women with initial surgery at an 
external facility by a non-CRN health plan-employed 
surgeon, women with breast cancer diagnosed on pro- 
phylactic mastectomy, and women with breast cancer 
diagnosed but never treated surgically. 

BRCASO database 

The BRCASO database is stored on a SQL server 2008 
database at VARI, the primary site for this research. 
Because of the multi-site nature of this study, we fol- 
lowed strict security procedures to ensure confidentiality 
of data transfer to VARI. Figure 1 shows electronic lim- 
ited data sets from each CRN institution were collated 
at a single site (Kaiser Permanente Colorado), before 
transfer to VARI through a Secure File Transfer Proto- 
col (SFTP) site using Secure Shell (SSH2) encryption. 
The University of Vermont transferred electronic data 
directly to VARI using the same SFTP site. Figure 2 
shows chart data were abstracted at individual institu- 
tions and entered into a secure (HTTPS) online web 
application hosted by VARI. VARI stored and collated 
all electronic and chart abstracted data behind their fire- 
wall, where they were only accessible to programmers 
and system administrators at VARI. 

Electronic administrative data 

Each CRN health plan collects electronic administrative 
data on inpatient and outpatient procedures, health plan 
enrollment, pharmacy dispensings, laboratory tests, and 
other elements as part of clinical care. Each health plan 
also links to their local Surveillance, Epidemiology and 
End Results (SEER) or state cancer registry. All electro- 
nic data elements for CRN sites are part of the Virtual 
Data Warehouse (VDW) where they are formatted in a 
standard fashion across sites, but the data themselves 
are housed at each individual site [5]. For BRCASO, a 
programmer at one site wrote programs to identify eligi- 
ble breast cancer cases and extract relevant administra- 
tive data that could be run against the VDW at each 
site. We extracted electronic information from: tumor 
registry data on diagnosis date, diagnosis type, laterality, 
estrogen and progesterone receptor status, and tumor 
grade; inpatient and outpatient Current Procedure Ter- 
minology (CPT-4) codes including date and type of pro- 
cedures (partial mastectomies, total mastectomies, and 
lymph node surgeries) with de-identified surgeon and 
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Initial electronic data transfer from remote sites to VAI 
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Figure 1 Shows how electronic data were transferred from each Cancer Research Network site (Marshfield and Group Health) to KPCO 
(Kaiser Permanente Colorado) before transfer to VARI (Van Andel Research Institute). Vermont transferred data directly to VARI. Data were 
routed through the VARI FTP (File Transfer Protocol) server and underwent initial validation before transfer to the BRCASO database. 



facility IDs; and demographic data on age, height, 
weight, geocoded income and education, ethnicity, race, 
and insurance type. These data were used to prefill the 
BRCASO database to reduce the amount of information 
that needed to be collected manually from the medical 
records. 

Abstracted medical record data 

Each site reviewed their own electronic medical 
records for women eligible for the study. All abstrac- 
tors had previous experience abstracting cancer 
patients' medical records. We developed a detailed 
data manual to standardize abstracted information 
(Additional file 1). This comprehensive manual pro- 
vided not only data definitions, but also preferences 
for which clinical report information was generally 
most accurate and an overall context of how the data 



element would be used in this study. Chart abstraction 
was piloted at each institution using paper forms and 
results for 10 cases from each site were discussed via 
teleconference before finalizing the manual and data- 
base. All electronic pre-filled data were viewable within 
the database, and could be edited if discrepancies were 
noted between administrative and chart data. Each 
patient was automatically marked as complete once 
every data field was complete, and the record was then 
routed to a SQL database at VARI. 

Clinical details abstracted from medical records included 

♦ Surgical breast procedures (multiple records per per- 
son allowed): indication (known malignancy vs. undiag- 
nosed breast lesion), anesthesia type, tumor localization 
technique, intra-operative margin assessment, surgeon 
specimen orientation technique, breast reconstruction 
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Chart abstraction data input from remote sites to VAI 
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Figure 2 Shows how chart abstraction data were collected from each Cancer Research Network site (KPCO [Kaiser Permanente 
Colorado], Marshfield, and Group Health) and Vermont and entered into a secure (HTTPS) online web application hosted by VARI. 

Data underwent initial validation before transfer to the BRCASO database. 



♦ Surgical margins (collected per procedure): direction 
and distance of closest uninvolved margins by invasive 
carcinoma and/or DCIS, direction of margins involved 
(i.e. positive margin) by invasive carcinoma and/or DCIS 

♦ Tumor information: tumor sizes (pathologic and as 
estimated by imaging prior to first treatment), size of 
invasive and DCIS components, presence of residual 
tumor in first and subsequent surgical specimens, syn- 
chronous tumors, final pathological diagnosis, lympho- 
vascular invasion, perineural invasion, HER2 result, 
prophylactic mastectomy pathology 

♦ Lymph nodes: type of procedure (sentinel lymph 
node [SLN] biopsy, axillary sampling/dissection), intra- 
operative SLN evaluation, SLN histology, SLN and axil- 
lary positive and total node counts 



♦ Other: Neoadjuvant treatment, imaging performed, 
known multi-focal and multi-centric status pre-opera- 
tively, prior breast cancer history 

Quality control 

Each chart abstractor was trained by the lead chart 
abstractor at the University of Vermont. Regular 
abstractor meetings occurred throughout the funding 
period to address specific questions. Chart abstractors 
could also post questions to a private SharePoint site 
that were answered by the lead chart abstractor or prin- 
cipal investigator and then disseminated to all abstrac- 
tors for review. After completing 1,000 charts, we 
reviewed frequencies of BRCASO data elements by insti- 
tution to check for inconsistencies, missing data, or 
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distributions differences by institution. A report was 
provided back to each site, where abstractors confirmed 
any inconsistencies and made changes to the BRCASO 
database if needed. We continued to review frequencies, 
perform variable cross-checks, and clean data as neces- 
sary throughout the abstraction period. The database 
was designed to keep a detailed log of all entries and 
edits to the data by user ID to maintain data integrity. 

Results 

We have complete data on 4524 of 6095 women (74.2%) 
and their surgeries in the BRCASO database, to date. 
The mean age was 61 years (range 20-98 years) and 94% 
of the study population was white. Approximately 40% 
of the study population were insured by Medicare and 
4% by Medicaid. Nearly 70% of women were diagnosed 
with IDC, 20% with DCIS, and 10% with ILC. 

By using electronic administrative data, CRN health 
plans pre-filled 22 of 72 (30%) data fields. The remain- 
der of the 50 fields required abstraction from the medi- 
cal record. In addition, several electronic data fields 
occasionally required editing by the chart abstractor to 
update and clarify diagnosis and procedure dates, per- 
forming surgeon, surgical facility, and lymph node pro- 
cedures. The total time for chart abstraction ranged 
from 30-90 minutes per chart depending on the number 
of surgeries per woman, and the availability of pathology 
and operative reports within the medical record. A list 
of potential quality measures that will be evaluated 
using these data for variation across women, surgeons, 
and facilities are in Table 1. We specifically selected 
eight measures that we felt would be meaningful to 
patients and were previously evaluated in the VBCSO 
database; [4] we included two additional measures (use 
of needle biopsy for pre-operative diagnosis, and use of 
neoadjuvant chemotherapy for T2/T3 tumors) that 
showed broad variation during data collection. In gen- 
eral, these were regularly recorded and easily found in 



automated data or medical records, which increases the 
adaptability of our methods to other institutions. 

Discussion 

Developing quality measures for breast cancer surgery is 
critical given the high frequency of surgical breast pro- 
cedures and potential variation that exists across facil- 
ities and surgeons. Wasif et al. comment that quality 
measures must 1) be acceptable to various stakeholders; 
2) include measureable elements; and 3) impact out- 
comes [2]. While several national and international 
groups have developed quality cancer care measures, 
most focus on treatment other than breast surgery and 
participation in these programs tends to be voluntary. 
BRCASO aims to develop a more comprehensive, vali- 
dated set of breast cancer surgical quality measures, and 
use these data to conduct research on surgical quality 
variation among a generalizable (i.e. non-voluntary) 
population. 

Current quality measurement programs for cancer 
care delivery, while valuable, are limited in their ability 
to specifically address surgical quality and provide 
detailed data for research. Two of these programs, 
QOPI - ASCO's Quality Oncology Practice Initiative [6] 
and the American College of Surgeons' (ACoS) Com- 
mission on Cancer (CoC), [7] are primarily geared 
toward measuring and documenting overall quality of 
cancer care. Together, the CoC and American Cancer 
Society developed the National Cancer Data Base, one 
of the largest national registries of new cancer diag- 
noses. The data collected for each of these efforts pri- 
marily relate to documentation and treatment other 
than surgery. 

One recent program dedicated primarily to breast care 
quality improvement is the American Society of Breast 
Surgeons' Mastery of Breast Surgery Program, another 
voluntary initiative that helps surgeons document their 
clinical performance; early results showed program 



Table 1 Potential surgical quality measures to be evaluated using BRCASO data 

SURGICAL OUTCOME MEASURE 

PRIMARY TUMOR Mastectomy rate 

Positive/close margin rate following initial partial mastectomy 

Partial mastectomy re-excision rate 

Use of needle biopsy for pre-operative diagnosis 

Use of neoadjuvant chemotherapy for T2^3 tumors 

AXILLARY MANAGEMENT Number of sentinel nodes identified 

Intra-operative lymph node assessment 
Nodes examined following axillary dissection 

Completion axillary dissection rate following positive sentinel lymph node 



HEALTH CARE DELIVERY 



Initial breast cancer surgery within 30 days 
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participation rates among surgeons practicing in aca- 
demic settings was low [8]. Facilities may also receive 
certification from the National Accreditation Program 
for Breast Centers (NAPBC) [9]. Accredited breast cen- 
ters are required to meet certain surgical standards 
including > 50% of early stage breast cancer patients 
treated with breast conserving surgery, SNL for all 
patients with stage I or II disease, and diagnosis with 
palpation- or image-guided biopsy rather than open 
biopsy. 

Perhaps the most detailed surgical quality measures 
come from the National Consortium of Breast Centers, 
Inc. and its National Quality Measures for Breast Cen- 
ters (NQMBC) program [10]. The NQMBC's quality 
measures include 37 measures for breast cancer diagno- 
sis and treatment including surgical timeliness, pathol- 
ogy timeliness, pathology report completeness (tumor 
size, margins, lymph nodes, and specimen sampling ade- 
quacy), sentinel node biopsy, breast conservation surgery 
rate, and re-excision rate. The program includes over 
200 voluntary participants and allows them to compare 
their performance with other centers across the US. 
Members can collect all measures or a select few 
depending on their certification level. The validity of 
some of these measures as reasonable quality measures 
has not been tested in a broad population of patients. 

The data collected from the BRCASO study will 
enable us to evaluate a number of quality measures spe- 
cific to breast cancer surgery. Our three initial proposed 
surgical quality measures are 1) initial mastectomy rate, 
2) initial partial mastectomy positive margin rate, and 3) 
initial partial mastectomy re-excision rate. At this time, 
we are not recommending benchmarks for these mea- 
sures, but instead, will evaluate variability in these mea- 
sures by patient-, surgeon-, facility-, and regional-level 
factors, and by invasive and in situ diagnoses that may 
inform future benchmarks. We will be able to control 
these outcomes for clinical factors such as estimated 
tumor size, known multi-centric disease, breast imaging, 
anesthesia, localization technique, neo-adjuvant treat- 
ment, and detailed tumor characteristics. We also have 
detailed data on timeliness of procedures, extent of 
lymph node procedures, positive node counts, and 
reconstruction, which may serve as additional surgical 
quality measures. Most importantly, we have collected 
margin status for each procedure performed, which is 
important in assessing surgical quality and cannot be 
extracted from tumor registry data. 

The BRCASO database has the potential to be one of 
the largest, multi-site breast cancer surgical quality 
research databases to-date. The research possibilities 
may extend even beyond surgical quality measures. One 
of the major differences between BRCASO and the 
national efforts described above is that BRCASO does 



not rely on voluntary reporting. BRCASO includes a 
diverse geographic sample of all surgeons who practice 
at our varied institutions (health maintenance organiza- 
tion, university hospital, contracted community hospi- 
tals). In addition, a proportion of the data from the 
CRN sites were collected from electronic administrative 
data, a process that has the potential to be extended to 
other organizations with similarly organized billing data. 

However, the BRCASO database is not without limita- 
tions. Our electronic administrative data were less suffi- 
cient than we had hoped for in pre-filling the database. 
Thus our chart abstraction took longer than we antici- 
pated, and we fell short of the originally anticipated 
6,095 cases. We have no detailed information on sur- 
geon training, surgeon specialty, patient breast size, 
family history or patient preferences - all of which can 
account for variability in surgical procedures and qual- 
ity. The focus of BRCASO was to collect data on speci- 
fic short-term surgical outcomes such as reexcisions 
rates; thus we do not have data on long-term outcomes 
of recurrence or mortality, but plan to expand our 
efforts to collect these data in the future. 

We learned several important lessons that others may 
want to consider when collecting surgical quality data. 
First, while electronic administrative data were useful in 
prefilling some of our data elements, they have serious 
limitations. Most importantly, there is no way to get 
accurate detailed margin or lymph node data from elec- 
tronic administrative data; re-excision data may also be 
inaccurate as repeat procedures do not appear to be 
well documented in electronic data. Tumor registry data 
generally represent final surgical procedure and final 
pathology, but do not offer a clear characterization of 
multiple procedures that a patient may receive. The 
CoC database does collect information on positive/nega- 
tive margin status, but does not include detail on mar- 
gin distance or direction. Until more detailed, standard 
definitions are developed and implemented, we can only 
obtain detailed surgical quality data through medical 
record abstraction; but these data are time consuming 
and expensive to collect. We noted modest variation in 
the information provided in pathology, surgical opera- 
tive, and other clinical reports between organizations 
and facilities. While standardizing these reports across 
institutions would be ideal, it was not possible in this 
retrospective study. Therefore, having highly trained 
medical record abstractors, a detailed coding manual, 
and regular channels for abstraction-related queries 
were invaluable to reducing data discrepancies in our 
studies. 

Collecting a large amount of multi-site surgical quality 
data is a complex task - but the potential benefits and 
knowledge gained may be substantial. While some of 
the national efforts mentioned here will provide larger 
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databases with which to evaluate quality of breast cancer 
care, only one (the NQMBC program) appears to collect 
the level of detail needed to evaluate quality of surgical 
care. These databases also rely solely on self-reported 
information from providers who opt-in to each system, 
and may be more likely to involve high volume breast 
surgeons. These providers may not be representative of 
the national population of surgeons because many sur- 
geons performing breast surgery in the United States are 
neither high volume breast surgeons nor work within a 
certified breast center [8]. Our study collected data on 
every surgeon who performs breast procedures at one of 
four geographically disparate institutions, with variable 
volume in breast surgery. Standardizing clinical data col- 
lection and reporting efforts across programs and data- 
bases (including ours), with institutional participation 
rather than voluntary, and publicly reporting results will 
likely be necessary to develop timely benchmarks for 
surgical quality. Additional programming work, such as 
Natural Language Processing, may help improve data 
collection timeliness and efficiency by automatically 
interpreting text from electronic medical records, thus 
reducing the need for manual abstraction. Creating stan- 
dard templates in electronic medical records for pathol- 
ogy and surgery reports may be another method that 
can improve both quality and consistency of data col- 
lected and reported. We hope the work from BRCASO 
provides a stepping stone for research on and develop- 
ment of standard surgical quality measures that can be 
generalized to and implemented by other providers and 
institutions. 

Additional material 



Additional file 1: BRCASO chart abstraction manual. Detailed chart 
abstraction methods and protocol used in the BRCASO study. 
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